Web Survey Bibliography
We describe a methodology for combining a convenience sample with a probability sample in a way that seeks to minimize the total mean squared error (MSE) of the resulting estimator. We then explore the properties of the resulting composite estimator, a linear combination of the convenience and probability sample estimators with weights that are a function of bias. We discuss the estimator’s properties in the context of web-based convenience sampling. Our analysis demonstrates that the use of a convenience sample to supplement a probability sample for cost-effective improvements in the MSE of estimation may be practical only under very limited circumstances. First, the bias remaining, after steps are taken to reduce it, must be quite small, equivalent to no more than 0.02-0.1 standard deviations (about one to five percentage points for a dichotomous outcome). Second, the probability sample should contain at least 1000-10,000 observations in order to effectively estimate bias. Third, it must be inexpensive and feasible to collect at least thousands (and probably tens of thousands) of web-based convenience observations. The convenience sample may be a useful supplement in a large survey where the primary goal is estimates within smaller domains if one is willing to assume that global bias estimates from the full sample also apply to smaller domains. The conclusions about the limited usefulness of convenience samples with estimator bias of more than 0.1 standard deviations can be shown to apply more generally.
Homepage (abstract)
Web survey bibliography - WAPOR 61th Annual Conference, 2008 (55)
- Testing the Effects of Multiple Manipulations on Print and Online Survey Response Rates: Lessons Learned...; 2008; Bachman, M., Vaccaro, D.
- The Role of Cash Incentives in Online Panelist Motivations: Experimental Results on Unit Response and...; 2008; Taylor, E.
- ‘For Example’: How Different Example Types in Online Surveys Influence Frequency ; 2008; Berent, M., Krosnick, J. A.
- Validating Check-All and Forced-Choice Question in a Paper Survey of Provincial Park Campground Users...; 2008; Dyck, B., Moore, D.
- Measuring Attentiveness to Current Events in a Mixed Mode Experiment; 2008; Suls, R., Horowitz, J.
- Transitioning from Self-Reports to Self-Installed Electronic Audience Measurement; 2008; Trussell, N., Vanno, L., Matthess, E., Bailey, J., Link, M. W.
- The Role of New Technology and its’ Effect on Best Practices Methodology; 2008; Kendall, E.
- Rate of Response in Web-Based Data Collection as a Factor of Author of E-mail Invitation; 2008; Mitra, A.
- Graduate vs. Undergraduate Student Respondent Behavior Differences in Web Surveys; 2008; Showen, S., Eisenberg, D., Roe, D. J.
- Mode Effects and Non-Response Bias in an Undergraduate Student Satisfaction Survey: Results from a Randomized...; 2008; Beach, S., Musa, D., Beeson, P., Sparks, C.
- Worth the Weight?: The Benefits and Pitfalls in Applying Survey Weights to Web Surveys of College Undergraduates...; 2008; Bloom, J. D.
- Improving the Efficiency of Web Survey Experiments; 2008; Luks, S., Rivers, D.
- When Encouraging Looks Go Too Far: Using Virtual Humans to Understand the Role of Rapport in the Survey...; 2008; Foucault, B., Aguilar, J., Cassell, J., Miller, P. V.
- Social Cues Can Affect Answers to Threatening Questions in Virtual Interviews; 2008; Lind, L. H., Schober, M. F., Conrad, F. G.
- How Animated Agents Affect Responses in Open-Ended Interviews about Alcohol Use; 2008; Person, N. K.
- Virtual Interviews on Mundane, Non-Sensitive Topics: Dialog Capability Affects Response Accuracy More...; 2008; Conrad, F. G., Schober, M. F., Jans, M., Orlowski, R. A., Nielsen, D.
- Mall-Intercept vs. Online Panel: Does Sample Source for an Experimental Study Matter?; 2008; Lin, C. T. J.
- Representativeness in Online-Surveys Through Stratified Sample; 2008; Blasius, J.
- Comparing the Results of Probability and Non-probability Telephone and Internet Survey Data; 2008; Wang, R., Krosnick, J. A.
- Evaluating the Potential Contributions of a Web-based Convenience Sample to the Accuracy of a Probability...; 2008; Elliott, M. N., Haviland, A.
- “R U in the Network?!” Using Text Messaging Interfaces as Screeners for Working Cell Phone...; 2008; Buskirk, T. D., Rao, K., Callegaro, M., Arens, Z., Steiger, D. M.
- Sampling & Weighting Cell Phone Samples to Supplement RDD Surveys; 2008; Brick, J. M., Edwards, W. S., Lee, Sunghee
- Using the ESRC Question Bank: An Online Resource Developed for the Social Survey Research Community; 2008; Gibbs, J. C.
- Why Text Mine?; 2008; Parry, J., Tomashek, S.
- Internet Access Panels and Public Opinion and Attitude Estimates; 2008; Piekarski, L., Galin, M., Baim, J., Frankel, M. R., Augemberg, K., Prince, S.
- Combining Mail and Internet Methods to Conduct Household Surveys of the General Public: A New Methodology...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., Oneill, A.
- Experiment on Use of Internet Cell Phone Only Panelists to Supplement RDD Samples; 2008; Turakhia, C., Schulman, M. A., Bohinsky, S.
- Evaluating Efficiency and Effectiveness of Cell Phone Samples; 2008; Sen, S., Zmud, J., Arce, C.
- Does the Inclusion of Mail and Web Alternatives in a Probability-Based Household Panel Improve the Accuracy...; 2008; Rookey, B. D., Dillman, D. A., Hanway, S.
- The “Professional Respondent” Problem in Web Surveys; 2008; Rivers, D.
- Predictors and Barriers to Collecting Data from Early Childhood Educators Using the Web; 2008; Caspe, M., Sonnenfeld, K., Meagher, C., Sprachman, S., Scaturro, G.
- The MacroPoll Wireless Experience: Development and Lessons Learned.; 2008; Austin, J. D., Zullwack, R., Dyer, A., Dayton, J. J.
- My Cell Phone’s Ringing, “Caller Unknown,” Now What? Usage Behavior Patterns Among...; 2008; Buskirk, T. D., Rao, K., Kaminska, O.
- Pilot Development of a Smartphone-Enabled Full-Probability Panel; 2008; Hill, C., Biemer, P. P., Coombs, D., Eyerman, J.
- A Test of Short versus Long Cell Phone Interviews; 2008; Jones, Je.
- Evaluating the Characteristics of Landline User’s Intention to Switch to Cell Phone Only Use for...; 2008; Sanderson, M., Immerwahr, S., Eisenhower, D., Konty, K.
- Coverage Bias in Surveys Excluding Cell Phone Only Adults: Evaluation of Bias and Effectiveness of Post...; 2008; Peytchev, A., Carley-Baxter, L. R., Black, M. L.
- Landline and Cell Phone Usage Patterns Among Young Adults; 2008; Currivan, D. B., Roe, D. J., Stockdale, J.
- Practical Steps to Conducting Cellular Telephone Surveys; 2008; Howes, C., DeBello, A., Wolter, K., Wooten, K.
- Health Policy Concerns and Policy Preferences: A Comparison of Landline RDD and Cell Phone Only (and...; 2008; Zukin, C., Cantor, J., Brownlee, S., Boyle, J.
- Measuring Health in RDD Surveys: Are Estimates that Exclude the Cell-Only Population Accurate?; 2008; Freedner, N., Holterman, L. A., Hannah, K.
- Does Including Cell-Only Households in an RDD Survey Change the Estimates? The Case of the American...; 2008; Bryant, B. E., Baker, R. P.
- The Effects of Excluding Cell-Only Respondents on Understanding Religion in the United States; 2008; Smith, G. A., Cox, D., Pond, A.
- Use of FedEx: Early, Late or Never?; 2008; Pope, D.
- When is the Best Time to Invite a Respondent? An Analysis of E-mail Invitation Timing and Response to...; 2008; Sinibaldi, J., Hansen, S. E.
- Instant Messaging: Applicability for Contacting Potential Web Respondents?; 2008; Cox, C. J., Harwood, P. G., Swanhart, M.
- E-mail and Postcard Invitation Designs to Maximize Web-Survey Responses Rates; 2008; Kaplowitz, M. D., Lupi, F., Couper, M. P., Thorp, L.
- Latent Class Modeling in Survey Methods: Estimation of the Cell Phone Only Population; 2008; Albaghal, M.
- Calculating Response Rates for Cell Telephone Surveys; 2008; Barron, M., Khare, M.
- Predicting Survey Bias in a Brave New Mobile World: Using the Behavioral Theory of Lifestyle Adoption...; 2008; Ehlen, P., Ehlen, J.